Efficient and low-cost fault tolerance for web-scale systems

s away rate-limiting issues. Work on DoS tolerance subsequent to the work of this thesis has developed DoS-tolerance mechanisms for homogeneous Byzantine models [ACKL08; CWA09]. 4.4.2 Normal Operations This section describes runs where there is no message loss and there is a single correct leader coordinator which has completed recovery and which is trusted by all correct coordinators. In these runs, the protocol proceeds through three phases upon the reception of a request from a client (see Figure 4.2). In Phase 1, it tries to provide a quick answer to the client. In Phase 2, it goes through an additional coordination step to let coordinators and servers know about the reply (potentially) delivered by the client. Finally, in Phase 3 it ensures that slow servers can directly retrieve old requests from at least one coordinator without triggering other instances of the agreement protocol. Phase 1: Replying to clients. When the client wants the service to perform an operation op, it initiates the protocol by sending a REQUEST message to all coordinators (lines 4.1–4.3). Clients periodically resend pending requests to the coordinators (lines 4.11–4.13). When the leader coordinator receives a request (line 5.1), it forms a proposal by attaching a proposal number prop to the request. The proposal number is used by the other coordinators to discard messages coming from old leaders. Each coordinator is assigned a partition of the set of positive 4.4. THE HETERTRUST PROTOCOL 55 Algorithm 4: Client cl upon invoke(op) 4.1 send (REQUEST, op) to all coordinators; 4.2 start timer; 4.3 4.4 upon receive (ACCEPTED, k, op, prop, repl) message from coordinator co 4.5 if op is pending then 4.6 if received (ACCEPTED, k, op, prop, repl) from d(c+ 1)/2e coordinators then 4.7 stop timer; 4.8 deliver(repl); 4.9 4.10 upon timeout 4.11 resend the pending REQUEST message to all coordinators; 4.12 start timer; 4.13